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METHODS OF PRODUCING SECURITY with 300 dpi scanners available for under $100, and 600 dpi 

DOCUMENTS HAVING DIGITALLY scanners available for marginally more. Similarly, photo- 

ENCODED DATA AND DOCUMENTS graphic quality color ink-jet printers are commonly available 

EMPLOYING SAME from Hewlett-Packard Co., Epson, etc. for under $300. 

5 These tools pose new threats. For example, a banknote 

RELATED APPLICATION DATA can be doctored (e.g. by white-out, scissors, or less crude 

techniques) to remove/obliterate the visible patterns on 

This application claims benefit of the Apr. 16, 1998, filing which prior art banknote detection techniques relied to 

date of co-pending provisional application No. 60/082,228. prevent counterfeiting. Such a doctored document can then 

This application is also a continuation-in-part of application 10 be freely scanned or copied, even on photocopiers designed 

Ser. No. 08/967,693, filed Nov. 12, 1997 (now Patent to prevent processing of banknote images. The removed 

6,122,392), which is a continuation of application Ser. No. pattern(s) can then be added back in, e.g. by use of digital 

08/614,521, filed Mar. 15, 1996 (now U.S. Pat. 5,745,604), image editing tools, permitting free reproduction of the 

which is a continuation of application Ser. No. 08/215,289, banknote. 

filed Mar. 17, 1994, now abandoned, which is a In accordance with aspects of the present invention, these 
continuation-in-part of application Ser. No. 08/154,866, and other current threats are addressed by digitally water- 
filed Nov. 18, 1993, now abandoned. This application is also marking banknotes, and equipping devices to sense such 
a continuation-in-part of application Ser. No. 08/951,858, watermarks and respond accordingly, 
filed Oct. 16, 1997 (now Patnet 6,026,193), which is a (Watermarking is a quickly growing field of endeavor, 
continuation of application Ser. No. 08/436,134, filed May ^ several different approaches. The present assignee's 
8, 1995 (now U.S. Pat. No. 5,748,763), which is a work fe reflected in the earlier-cited related applications, as 
continuation-in-part of application Ser. No. 08/327,426, ^"^S V ?-J* X ~£? S ' 5 >* 4 }>?] S > 5 ' 74 *^ 3 ' 5 ' 7 1 10 ' 834 ' 
filed Oct. 21, 1994 (now U.S. Pat. No. 5,768,426), which is fl^^F^ 9 i f^P™ ^ application 
„ : n „L „f c« wx nsm<; ->eo W097/43736. Other work is illustrated by U.S. Pat. Nos. 
fit T f appl TT 08/215,289, 5,646,997, 5,659,726, 5,664,018, 5,671,277, 
filed Mar. 17, 1994, referenced above. 25 faim, S&i&i S fi&)>*l\ 5 j68&0, Sfili t 7Al\ 
FIELD OF THE INVENTION ^,574,962, 5,579,124, 5,581,500. 5 613,004, 5,629,770, 

5,461,426, 5,743,631, 5,488,664, 5,530,759, 5,539,735, 

The present application relates to the use of digital 4,943,973, 5,337,361, 5,404,160, 5,404,377, 5,315,098, 

watermarking in connection with paper currency and other 5,319,735, 5,337,362, 4,972,471, 5,161,210, 5,243,423, 

security documents. 30 5,091,966, 5,113,437, 4,939,515, 5,374,976, 4,855,827, 

4,876,617, 4,939,515, 4,963,998, 4,969,041, and published 

BACKGROUND AND SUMMARY OF THE foreign applications WO 98/02864, EP 822,550, WO 

INVENTION 97/39410, WO 96/36163, GB 2,196,167, EP 777,197, EP 

736,860, EP 705,025, EP 766,468, EP 782,322, WO 
The problem of casual counterfeiting of banknotes first 95/20291, WO 96/26494, WO 96/36935, WO 96/42151, 
arose two decades ago, with the introduction of color WO 97/22206, WO 97/26733. Some of the foregoing pat- 
photocopiers. A number of techniques were proposed to ents relate to visible watermarking techniques. Other visible 
address the problem. watermarking techniques (e.g. data glyphs) are described in 
US. Pat. No. 5,659,628 (assigned to Ricoh) is one of U.S. Pat. Nos. 5,706,364, 5,689,620, 5,684,885, 5,680,223, 
several patents noting that photocopiers can be equipped to ^ 5,668,636, 5,640,647, 5,594,809. 

recognize banknotes and prevent their photocopying. The Most of the work in watermarking, however, is not in the 

Ricoh patent particularly proposed that the red seal printed patent literature but rather in published research. In addition 

on Japanese yen notes is a pattern well-suited for machine to the patentees of the foregoing patents, some of the other 

recognition. U.S. Pat. No. 5,845,008 (assigned to Omron), workers in this field (whose watermark-related writings can 

and U.S. Pat. Nos. 5,724,154 and 5,731,880 (both assigned 45 by found by an author search in the INSPEC database) 

to Canon) show other photocopiers that sense the presence include I. Pitas, Eckhard Koch, Jian Zhao, Norishige 

of the seal emblem on banknotes, and disable a photocopier Morimoto, Laurence Boney, Kineo Matsui, A. Z. Tirkel, 

in response. Fred Mintzer, B. Macq, Ahmed H. Tewfik, Frederic Jordan, 

Other technologies proposed that counterfeiting might be Naohisa Komatsu, and Lawrence O* Gorman, 

deterred by uniquely marking the printed output from each 50 The artisan is assumed to be familiar with the foregoing 

color photocopier, so that copies could be traced back to the prior art. 

originating machine. U.S. Pat. No. 5,568,268, for example, In the present disclosure it should be understood that 

discloses the addition of essentially-imperceptible patterns references to watermarking encompass not only the assign- 

of yellow dots to printed output; the pattern is unique to the ee's watermarking technology, but can likewise be practiced 

machine. U.S. Pat. No. 5,557,742 discloses a related 55 with any other watermarking technology, such as those 

arrangement in which the photocopier's serial number is indicated above. 

printed on output documents, again in essentially- The physical manifestation of watermarked information 

imperceptible form (small yellow lettering). U.S. Pat. No. mos t commonly takes the form of altered signal values, such 

5,661,574 shows an arrangement in which bits comprising as slightly changed pixel values, picture luminance, picture 

the photocopier's serial number are represented in the pho- 60 colors, DCT coefficients, instantaneous audio amplitudes, 

tocopier's printed output by incrementing, or decrementing, elc . However, a watermark can also be manifested in other 

pixel values (e.g. yellow pixels) at known locations by fixed ways, such as changes in the surface microtopology of a 

amounts (e.g. +/-30), depending on whether the correspond- medium, localized chemical changes (e.g. in photographic 

ing serial number bit is a "1" or a "0." emulsions), localized variations in optical density, localized 

Recent advances in color printing technology have greatly 65 changes in luminescence, etc. Watermarks can also be 

increased the level of casual counterfeiting. High quality optically implemented in holograms and conventional paper 

scanners are now readily available to many computer users, watermarks.) 
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The foregoing and other features and advantages of the techniques) to remove/obliterate the visible structures. Such 

present invention will be more readily apparent from the a document can then be freely copied on either a visible 

following Detailed Description, which proceeds with refer- structure-sensing photocopier or scanner/printer installation, 

ence to the accompanying drawings. The removed visible structure can then be added in via a 

5 second printing/photocopying operation. If the printer is not 

BRIEF DESCRIPTION OF THE DRAWINGS equipped with banknote-disabling capabilities, image- 
editing tools can be used to insert visible structures back into 

FIG. 1 shows part of an automatic teller machine employ- mZ g e d ala ^ scanned from such doctored bills, and the 

ing principles of the present invention. complete bill freely printed. By additionally including 

FIG. 2 shows part of a device (e.g. a photocopier, scanner, Q embedded watermark data in the banknote, and sensing 

or printer) employing principles of the present invention. same, such ruses will not succeed. 

FIG. 3 shows part of another device employing principles ( A similar ruse is to scan a banknote image on a non- 

of the present invention. banknote-sensing scanner. The resulting image set can then 

be edited by conventional image editing tools to remove/ 

DETAILED DESCRIPTION J5 obliterate the visible structures. Such a data set can then be 

printed— even on a printer/photocopier that examines such 

Watermarks in banknotes and other security documents data for the presence 0 f visible structures. Again, the missing 

(passports, stock certificates, checks, etc.— all collectively visibIe structures ^ a be inserted by a subsequent printing/ 

referred to as banknotes herein) offer great promise to reduce photocopying operation ) 

such counterfeiting, as discussed more fully below Desirably , thc visible structurc det ector and the water. 

AddinonaUy watermarks provide a high-confidence tech- 20 ^ ^ ^ d ^ ^ & hafdware 

nique for banknote authentication. and/or tQol ^ arrangement provides various 

By way of example, consider an automatic teller machine economies, e.g., in interfacing with the scanner, manipulat- 

that uses watermark data to provide high confidence authen- mg p i xe i data xts f or pa ttern recognition and watermark 
tication of banknotes, permittinp; it to accept— as-werj-as- extraction, electronically re-registering the image to facili- 

dispepse— cash. Referringj p'FIG. 1, such a machine (11) is tate pat t era recognition/watermark extraction, issuing con- 

provided with a known optical scanner (13) to produce lrol s i gn als (e.g. disabling) signals to the photocopier/ 

digital data (15) corresponding to the face(s) of the bill (16). scanner, etc. 

This image set (14) is then analyzed (16) to extract embed- A rdated princip i e (pja 3) is to insert an imperceptible 
ded watermark data. In watermarking technologies that watermark having a universal ID (UID) into all documents 
require knowledge of a code signal (20) for decoding (e.g. primed wim a prmtef( with a ^ncr, or reproduced 
noise modulation signal, crypto key, spreading signal, etc.), by a photocop j er . Th e UID is associated with the particular 
a bill may be watermarked in accordance with several such printer/photocopier/scanner in a registry database main- 
codes. Some of these codes are public-permitting their tained by the producls » manufacturers. The manufacturer 
reading by conventional machines. Others are private, and can aJso enter in this data5ase the name of lhe distributor to 
are reserved for use by government agencies and the like. whom the product was initially shipped. Still further, the 
(Cf. public and private codes in the present assignee's owner > s name and a ddress can be added to the database 
issued patents.) when the machine is registered for warranty service. While 

As noted, banknotes presently include certain visible not preventing use of such machines in counterfeiting, the 
structures, or markings (e.g., the seal emblem noted in the ^ embedded UID facilitates identifying the machine that gen- 
earlier-cited patents), which can be used as aids to note erated a counterfeit banknote, (This is an application in 
authentication (either by visual inspection or by machine which a private watermark might best be used.) 
detection). Desirably, a note is examined by an integrated while the foregoing applications disabled potential coun- 
detection system (24), for both such visible structures (22), terfeiting operations upon the detection of either a visible 
as well as the present watermark-embedded data, to deter- 45 structure or watermarked data, in other applications, both 
mine authenticity. criteria must be met before a banknote is recognized as 

The visible structures can be sensed using known pattern genuine. Such applications typically involve the receipt or 

recognition techniques. Examples of such techniques are acceptance of banknotes, e.g. by ATMs as discussed above 

disclosed in U.S. Pat. Nos. 5,321,773, 5,390,259, 5,533,144, and illustrated in FIG. 1. 

5,539,841, 5,583,614, 5,633,952, 4,723,149 and 5,424,807 50 The foregoing principles (employing just watermark data, 

and laid-open foreign application EP 766,449. The embed- or j n conjunction with visible indicia) can likewise be used 

ded watermark data can be recovered using the scanning/ to prevent counterfeiting of tags and labels (e.g. the fake 

analysis techniques disclosed in the cited patents and pub- hbcls and tags commonly used in pirating Levis brand jeans, 

lications. branded software, etc.) 

To reduce counterfeiting, it is desirable that document- 55 The reader may first assume that banknote watermarking 
reproducing technologies recognize fran^pies and refose to is effected by slight alterations to the ink color/density/ 
reproduce same. Referring to pn 2. a photocopier (30), for distribution, etc. on the paper. This is one approach. Another 
example, can sense the presence oieithejLa_vjsfo le structure is t 0 watermark the underlying medium (whether paper, 
($2\ nr embedded banknote watermark data (34Y and ffes hle polymer, etc.) with a watermark. This can be done by 
copying if either is present (36). Scanners and printers can 50 changing the microtopology of the medium (a la mini- 
be equipped with a similar capability — analyzing the data Braille) to manifest the watermark data. Another option is to 
scanned or to be printed for either of these banknote hall- employ a laminate on or within the banknote, where the 
marks. If either is detected, the software (or hardware) laminate has the watermarking manifested thereon/therein, 
disables further operation. The laminate can be textured (as above), or its optical 
e watermark detection criteria provides an important 65 transmissivity can vary in accordance with a noise -like 
advantage not otherwise available. As noted, an original bill pattern that is the watermark, or a chemical property can 
can be doctored (e.g. by white-out, scissors, or less crude similarly vary. 
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Another option is to print at least part of a watermark For example, the technology and solutions disclosed 

using photo luminesce nt ink. This allows, e.g., a merchant herein have made use of elements and techniques known 

presented with a banknote, to quickly verify the presence of from the cited references. Other elements and techniques 

*some* watermark-like indicia in/on the bill even without from the cited references can similarly be combined to yield 

resort to a scanner and computer analysis (e.g. by examining 5 further implementations within the scope of the present 

under a black light). Such photoluminescent ink can also invention. Thus, for example-J nolograms with watermark 

print human-readable indicia on the bill, such as the denomi- ^data can be employed in banknotes, single-bit watermarking 

nation of a banknote. (Since ink-jet printers and other can commonly be substituted for multi-bit watermarking, 

common mass-printing technologies employ cyan/magenta/ technology described as using imperceptible watermarks 

yellow/black to form colors^ they can produce only a limited can a^^y ^ practiced ^ ng watermarks 

spectrum of colors^ Photoluminescent colors are outside (glyphs, etc.), techniques described as applied to images can 

their capamlmes. Fluorescent colors-^uch as the yellow, be applied to video and audio, local scahng of 

pink and green dyes used in highlighting markers — can . , < . . , . . & . 

similarly be used and have ihe advantage of being visible wate ™ a * cner 8y «» P'? v » ded . to enhance watermark 

- m—i, r-u. \ signal-to-noise ratio without increasing human 

without a black light.) ° cu • .• . , . 
.. " ' . . .. .. ..15 percepubihty, vanous filtering operations can be employed 

An improvement to exiting eroding techniques is to add , Q xfve ^ explained m the prior art 

an iterative assessment of the robustness of the mark, with can include subliminal graticules to aid in ■ 

a corresponding adjustment in a re-watermarking operation. re „ BgislIlliollf encoding may proceed at the granularity 0 fa 

Especially when encoding multiple bit watermarks, the , . w n rT a „ ■ , , 

, r - r. ii- , . single pixel (or DCT coefficient), or may similarly treat 

characteristics of the underlying content may result in some adjoinj rf ixeb (m Dcr the encod . 

bits being more robustly (e.g. strongly) encoded than others. . can be 0 timized t0 wit^d expected forms 0 f content 

in an illustrative technique employing this improvement, a cornlption . Etc 

, etc., etc. Thus, the exemplary embodiments 

watermark is first emb«lded m an object. Next, a trial afe on , ^ of me fc 

decoding operation is performed. Aconfidence measure (e * combiai the teachin referenced above . other ^ 

s.gnal-to-noise ratio) associated with each bit detected in the , ions necessaril ^ noI exbaustively describcd heteia> but 

decoding operation ,s then assessed. THe bits that appear afe faif , ^ , he of an artisan ^ ven the 

weakly encoded are tdenHfied, and corresponding changes f ^ djsclosure and familiarit ^ ^ ci(ed art . 

are made to the watermarking parameters to bring up the ,~ . , , . . , . , , 

relative strengths of these bil. The object is then water- , ^° pr0VM * a comprehensive disclosure without unduly 

marked anew, with the changed parameters. Tlis process can len g| heMn g the fi>n°wi«W specification, applicants incorpo- 

. - j j j 7. f 4 , . .„ r . . t . 30 rate by reference the patent documents cited herein.) 

be repeated, as needed, until all of the bits comprising the j 

encoded data are approximately equally detectable from the *** LJJ p , • . * ,. , . . , . 

encoded object, or meet some predetermined signal-to-noise 1 ^ ° f u P * ? 

ratio threshold encoded therein, the method comprising: slightly altering an 

™ - ... , „ original image but without leaving any substantially human- 

The foregoing applications, and others, can generally 35 , , evidence of ^ alterati ^ rintin the 

benefit by muh.ple watermarks. For example, an object bankDOte ^ ^ ; whcrein ^ y ^ 

(physical or data) can be marked once m the spaual domain a - of , he banknote jdds ^ ^ f(0m wbich 

and a second time in the spatial frequency domain. (It should , .f Mn Ua a^^a^a <• ™„ a , *u 

. JtJiL . l • j • i_ data can De decoded, yet rendermg of the scan data for 

be understood that any change in one domain has repercus- .„„,„ •„ „• , c n . i avr . 1anM n c a 

. * wv f ... . . l ■ L L human viewing does not reveal the existence of said 

sions in the other. Here we reference the domain in which the ^ encoc i et j digital data 

change is directly effected.) 2 ^ memo(J Qf clajm 1 m ^ ^ da(a 

Another option is to mark an object with watermarks of comprises plural bits 

two different levels of robustness, or strength. The more 3 nc melhod of claim 2 m which ^ lural bits are 

robust watermark withstands various types of corruption, encoded ^od^y across the banknote , rat her than the 

and is detectable m the object even after multiple genera- 45 banknole b^g marked in a single localized region only. 

Hons of intervening distortion. The less robust watermark 4 ^ method of claim x ^ which the CDCoding makes 

can be made frail enough to fail with the first distortion of use of a code signal 

the object. In a banknote, for example, the less robust 5 ^ method of claim x m whicQ the encoding makes 

watermark serves as an authentication mark. Any scanning use of a discrete co Sme transform, 

and reprinting operation will cause it to become unreadable. 50 6 ^ method of cIaim ± which mcludes eacoding 

Both the robust and the frail watermarks should be present ^ digitaJ wate r m arks. 

m an authentic banknote; only the former watermark wUl be 7 llie method of claim 6 h which me tw0 differeQt ^^j^ 

present m a counterfeit. watermarks are of different robustness. 

Still another form of multiple-watermarking is with con- 8. The method of claim 6 in which the two watermarks are 

tent that is compressed. The content can be watermarked 55 encoded in accordance with different code signals, 

once (or more) in an uncompressed state. Then, after 9. The method of claim 1 which also includes providing 

compression, a further watermark (or watermarks) can be lne banknote with a hologram. 

applied. iq Tjj e method of claim 1 which includes encoding a 

Still another advantage from multiple watermarks is pro- calibration signal with the digital data, 

tection against sleuthing. If one of the watermarks is found 60 U. The method of claim 10 in which the calibration signal 

and cracked, the other watermark(s) will still be present and is adapted to facilitate decoding of the digital data from the 

serve to identify the object. encoded banknote notwithstanding rotation. 

The foregoing discussion has addressed various techno- 12. A method of enhancing the security of a banknote, the 

logical fixes to many different problems. Exemplary so hi- method including digitally watermarking a banknote with 

lions have been detailed above. Others will be apparent to 65 machine readable, generally imperceptible, digital data, 

the artisan by applying common knowledge to extrapolate characterized by generating a pattern corresponding to said 

from the solutions provided above. digital data, and physically texturing the surface of the 
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banknote in accordance with said pattern, said texturing 
being independent of printing on the banknote. 

13. The method of claim 12 in which said digital data 
comprises plural bits. 

14. The method of claim 13 in which said plural bits are 
encoded redundantly across the banknote, rather than the 
banknote being marked in a single localized region only. 

15. The method of claim 12 in which the encoding makes 
use of a code signal. 

16. The method of claim 12 in which the encoding makes 
use of a discrete cosine transform. 

17. The method of claim 12 which includes encoding with 
two different digital watermarks. 

18. The method of claim 17 in which the two different 
digital watermarks are of different robustness. 

19. The method of claim 17 in which the two watermarks 
are encoded in accordance with different code signals. 

20. The method of claim 12 which also includes providing 
the banknote with a hologram. 

21. The method of claim 12 which includes encoding a 
calibration signal with the digital data. 

22. The method of claim 21 in which the calibration signal 
is adapted to facilitate decoding of the digital data from the 
encoded banknote notwithstanding rotation. 

23. The method of claim 12 in which visible light scan- 
ning of the banknote yields scan data from which the digital 
data can be decoded, yet rendering of the scan data for 
human viewing does not reveal the existence of said 
encoded digital data. 

24. A method of producing a security document having 
digital data encoded therein comprising: slightly altering an 
original image, said alterations varying across the image in 
accordance with local image characteristics rather than 
being uniform thereacross, and printing the security docu- 
ment with the altered image, wherein visible light scanning 
of the security document yields scan data from which the 
digital data can be decoded, yet rendering of the scan data 
for human viewing does not reveal the existence of said 
encoded digital data. 

25. A method of producing a security document having 
digital data encoded therein, the method comprising: slightly 
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altering an original image but without leaving any substan- 
tially human- apparent evidence of image alteration, and 
printing the security document with the altered image, 
wherein visible light scanning of the security document 

5 yields scan data from which the digital data can be decoded, 
yet rendering of the scan data for human viewing does not 
reveal the existence of said encoded digital data. 

26. The method of claim 25 in which the digital data 
comprises plural bits. 

10 27. The method of claim 26 in which said plural bits are 
encoded redundantly across the security document, rather 
than the security document being marked in a single local- 
ized region only. 

]5 28. The method of claim 26 in which the encoding makes 
use of a code signal. 

29. The method of claim 26 in which the encoding makes 
use of a discrete cosine transform. 

30. The method of claim 26 which includes encoding with 
2 q two different digital watermarks. 

31. The method of claim 30 in which the two different 
digital watermarks are of different robustness. 

32. The method of claim 30 in which the two watermarks 
are encoded in accordance with different code signals. 

25 33 . The method of claim 25 which also includes providing 
the security document with a hologram. 

34. The method of claim 25 which includes encoding a 
calibration signal with the digital data. 

35. The method of claim 40 in which the calibration signal 
30 is adapted to facilitate deconding of the digital data from the 

encoded security document notwithstanding rotation. 

36. The method of claim 25 wherein the security docu- 
ment comprises a passport. 

37. The method of claim 25 wherein the security docu- 
35 ment comprises a check. 

38. The method of claim 25 wherein the security docu- 
ment comprises a lable. 

39. The method of claim 25 wherein the security docu- 
ment comprises a tag. 

40 

***** 
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